Supervised training sample selection for the estimation of spectral reflectance using a RGB camera

Spectral imaging can provide spectral information from which spectral radiance or reflectance can be recovered at each image pixel. Recovery algorithms lead to good spectral and colorimetric performance by directly transforming RGB digital counts to spectral reflectances, but his approach is sensitive to the size and composition of the training set. What we propose here is a supervised method to select the most appropriate samples from a training database to buld the transformation matrix relating digital counts to spectral reflectances. Thus, this approach is tested with real images. Introduction Multispectral imaging uses a digital camera coupled or not with colour filters of different spectral bands, ranging from just one band, as in a monochromatic system, to hundreds of components, as in a ultraspectral system [1]. The main advantage of spectral imaging in comparison with conventional spectroradiometric measurements is that they can provide spectral information from which spectral radiance or reflectance can be recovered at each image pixel. During the last years, many different approaches have been proposed for spectral reflectance recovery [2]. Among these techniques, recovery algorithms lead to good spectral and colorimetric performance by directly transforming RGB digital counts to spectral reflectances [3]. This approach can be sensitive to the size and composition of the training set of reflectances [4]. Other authors have shown that depending on the spectral application (e. g. spectral pigment analysis in art pictures) the quality of spectral recovery may change dramatically. In this work we propose a supervised training sample selection method for spectral reflectance estimation based on a linear pseudo-inverse algorithm. The method uses a set of training samples selected from a database which are in the neighborhood of the target sample to optimize the recovery matrix relating digital counts to spectral reflectances. What we propose here is to optimize the building process of the transformation matrix using a learningbased algorithm. Recently different computational results suggest that using cut off filters to recover reflectances or illuminants may or not improve the recovery performance, depending on the presence of noise [5, 7]. As these results are not clear for noisy data, we will also analyze here the effects of adding successive cut off filters when natural scenes are captured using a real RGB camera. Method In the present work, we used real images captured with an RGB digital color camera from QImaging (model Retiga 1300, 12 bits). The images captured were the Gretag Macbeth Color Checker DC and Color Checker rendition chart [8]. In a second step we added cut off filters in front of the camera lens (GG475 and OG550, from OWIS GMBH) to study possible improvements. Given a set of training spectra S (which can be spectral radiances or reflectances) and the corresponding set of experimental camera responses ρ, a recovery transformation matrix D is defined by D = Sρ, where ρ is the pseudo-inverse of ρ. If ρ has full rank, then ρ = (ρ ρ) ρ, where ρ is the transpose of ρ. An estimate of Ŝ1 of a set of test spectra S1 may then be obtained from the corresponding set of camera responses ρ1 by applying the transformation D, that is Ŝ1 = Dρ1 [7]. We present here a method to select the most appropriate samples from a training database to be used as training set to recover reflectances from RGB data of a test image, without any need of knowing any spectral information of the test sample. The first step is to calculate CIELAB coordinates from both test and training set using real RGB digital counts captured with a CCD camera in both cases. Once we have this information, CIELAB color difference di is calculated between each test sample and the whole training set. It allows us to sort the training set for each test sample from minimum to maximum color difference. To choose the training set elements to recover reflectance for each test sample, we set a sphere in each element of the test set in the CIELAB space, and we will use the elements of the training set inside this sphere. To calculate the radius of this sphere, we implement an iterative process as explained in Figure 1. It sets an initial sphere in the test sample, looks for the elements of the training set inside this sphere and compute the mean of its distances, giving them a weight that depends on the number of iterations and the number of elements inside the sphere. Then, the radius of the sphere is increased and the process is repeated decreasing the weight. The algorithm description is: Fix a starting radius r0, a constant k, and the number of iterations a. In this work the appropriate values of those constants comes from previous experimental results. Set a first sphere of radius r0 centered on a test sample in the CIELAB space, and look for the elements of the training set inside of the sphere. Those elements will have CIELAB differences di ≤ r0. Now, it is possible to calculate a weighted mean radius T0 as: